Technical Field
The present invention relates to signal processing, and more particularly to automatic discovery of message ordering invariants in heterogeneous logs.
Description of the Related Art
Some homogenous log analysis tools largely utilize mining algorithms to identify the most common/frequent log sequence patterns from logs and use such frequent patterns as a normal model for anomaly detection. Such homogeneous log analysis tools are typically customized to one single specific application or service. In particular, domain knowledge about the application/service and in particular the log formats/semantics are completely available and the homogenous log analysis tools fully utilize such knowledge. The problem with such homogeneous log analysis tools is that once the system providing the logs has updates, the homogeneous log analysis tools have to be manually updated.
Moreover, prior art approaches typically suffer from scalability issues and cannot be applied to arbitrarily large systems. In addition, anomaly detection performance is dependent on the sensitively of the system parameters, which makes the anomaly detection difficult to configure.
Some homogeneous log analysis tools provide an analysis over a system but with a strong bias regarding the nature of system behaviors (e.g., sequential ordering of certain events, causality relations among events, and so forth). Typically, prior knowledge about the system is accessible and thus the analysis is designed based on, or to conform to, such knowledge. Such systems also have limitations on their applicability to other systems of different or unknown natures.